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Summary 


This  report  describes,  in  general  terms,  the  results  of  basic  research  conducted  in  five 
areas  in  which  resourceful  computing  is  critical  for  defense  applications. 

•  Engineering  apprentices  for  resourceful  repair,  design,  and  specification. 

•  Sensing  prototypes  for  recognition,  navigation,  and  manipulation. 

•  Hardware  and  software  solutions  for  ultra-large  scale  computing  problems. 

•  Analogical  and  common  sense  reasoning  for  analysis  and  design. 

The  remainder  of  this  report  provides  a  bit  more  detail  in  each  area.  Fully  detailed 
accounts  of  the  research  results  arc  accessible  through  the  lists  of  technical  reports  and 
publications. 

Special  Publication  Note 

Much  of  the  work  performed  under  this  contract  was  reported  on  in  a  massive  two-volume 
collection  titled  Artificial  Intelligence  at  MIT:  Expanding  Frontiers,  edited  by  Patrick  H. 
Winston  with  Sarah  A.  Shellard  (1990).  Copies  are  available  through  the  MIT  Press. 

Engineering  Apprentices 


Model  Based  Reasoning  Systems 

Professor  Davis,  Dr.  Shrobc,  and  their  associates  have  built  knowledge- based  systems 
that  use  models  of  structure,  function,  and  causality  to  perform  a  wide  range  of  problem 
solving  and  reasoning  tasks.  The  systems  they  have  built  can  reason  about  how  a  device 
works  and  how  it  fails  in  a  manner  similar  to  an  experienced  engineer.  This  is  an 
imponant  advance  in  the  an  of  knowledge- based  systems  construction,  because  it  provides 
the  system  with  a  more  fundamental  understanding  of  the  device  than  is  possible  using 
traditional  approaches. 

In  particular,  the  members  of  Professor  Davis’s  group  have  built  a  troubleshooting 
system  that  deals  with  devices  that  include  memory  and  complex  time-dependent  behav¬ 
ior;  a  system  that  generates  diagnostics  from  a  circuit  description,  capable  of  generating 
tests  for  devices  considerably  more  complicated  than  those  handled  by  existing  test  gener¬ 
ators;  a  system  that  functions  as  an  assistant  in  design  for  testability;  a  system  that  designs 
devices  by  reasoning  from  fundamental  principles  of  qualitative  physics  and  qualitative 
mathematics;  a  system  to  demonstrate  how  a  program  can  learn  from  experience,  using 
two  different  forms  of  generalization  along  with  a  set  of  guidelines  that  indicates  when  to 
remember  and  generalize,  and  when  to  simply  re-derive  the  result;  and  a  system  capable  ^ 
of  designing  representations  for  an  interesting  class  of  analytical  reasoning  problems. 

Work  during  the  final  year  of  the  contract  produced  two  new  systems.  One  assists  ^ 
with  consensus  knowledge  acquisition,  the  task  of  assisting  two  or  more  expens  in  coming 
to  consensus  on  the  knowledge  base  necessary  for  a  specific  task.  The  second  explored 
the  fundamental  problem  of  model  selection:  how  does  an  engineer  decide  which  model 
(that  is,  which  approximation)  to  use  when  solving  a  problem.  I  OistributionT 
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Work  to  be  carried  on  in  the  future  is  focused  on  understanding  how  things  work 
in  a  variety  of  domains,  including  simple  mechanical  devices  like  four-bar  linkages,  and 
mechanistic  explanations  of  biological  phenomena.  Examples  of  understanding  include 
the  ability  to  produce  descriptions  of  device  behavior  from  a  description  of  their  structure, 
the  ability  to  predict  behavior  under  unusual  circumstances,  and  the  ability  to  redesign 
to  fit  those  new  circumstances. 


Engineering  Problem  Solving  and  Design 

Professor  Ulrich  and  his  students  have  worked  on  computational  tools  for  product  design 
and  manufacturing.  One  project  was  aimed  at  encoding  and  exploiting  product  develop¬ 
ment  information  as  it  is  discovered  and  modified  in  the  course  of  a  product  development 
effort.  This  research  is  being  conducted  in  the  context  of  the  actual  development  of  a 
new  power  tool. 

Another  project  focused  on  the  problem  of  enforcing  producibility  constraints  during 
the  specification  of  the  geometry  of  structural  parts.  This  work  was  motivated  by  the  needs 
of  large  engineering  projects,  commercial  airframe  development  for  example,  involving 
thousands  of  structural  pans. 


The  Programmer’s  Apprentice 

Dr.  Rich,  Dr.  Waters,  and  other  members  of  the  Programmer’s  Apprentice  group  used 
programming  as  a  domain  for  studying  and  attempting  to  duplicate  human  problem  solv¬ 
ing  skills.  They  produced  a  system,  called  the  Programmer’s  Apprentice,  which  provides 
intelligent  assistance  in  various  phases  of  the  programming  process.  During  the  final 
year  of  the  contract,  they  completed  a  demonstration  of  automated  program  optimization 
(using  a  library  of  reusable  software  abstractions).  They  also  worked  on  automated  re¬ 
verse  engineering  (reconstructing  the  design  of  a  program  from  just  the  source  code)  and 
intelligent  assistance  for  software  design. 


Sensing  Prototypes 

Object  Isolation  and  Identification 

Work  directed  by  Professor  Ullman  explored  the  problem  of  three-dimensional  object 
recognition  without  three-dimensional  models.  This  research  was  divided  into  two  main 
topics.  The  first  topic  was  that  of  image  partitioning  and  selection.  The  goal  of  this 
processing  stage  is  to  select  from  the  image  a  portion  that  is  likely  to  contain  an  object 
of  interest.  The  selection  process  gives  the  recognition  system  a  capacity  that  is  similar 
to  the  use  of  selective  attention  in  human  vision:  it  allows  the  system  to  concentrate  its 
computational  resources  on  the  selected  structure  and  apply  to  it  additional  processing 
stages  that  will  lead  eventually  to  recognition.  Professor  Ullman  has  developed  a  method 
for  grouping  together  image  edges  and  contours  that  are  likely  to  correspond  to  a  single 
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object.  This  method  appears  to  capture  some  basic  properties  of  the  grouping  processes 
used  by  the  human  visual  system. 

The  second  topic  was  the  representation  of  three-dimensional  objects  in  memory, 
and  the  matching  of  these  memory  models  with  two-dimensional  objects  in  the  image. 
Towards  this  end.  Professor  Ullman  has  developed  two  approaches.  The  first,  called  the 
alignment  method,  finds  and  compensates  for  the  transformations  separating  the  viewed 
object  and  a  given  stored  model  prior  to  a  matching  operation.  The  second  approach  is 
novel  in  that  objects  are  recognized  without  storing  any  three-dimensional  object  models. 
Instead,  objects  arc  recognized  by  using  combinations  of  two-dimensional  views.  The 
method  is  based  on  a  theory  that  shows  that  any  view  of  a  three-dimensional  object  can 
be  approximated  by  the  linear  combination  of  a  small  number  of  its  views. 

Other  work  on  object  recognition,  directed  by  Professor  Crimson,  was  centered 
on  the  development  of  systems  for  recognizing  objects  in  cluttered,  noisy,  unstructured 
environments.  Such  systems  have  been  demonstrated  in  a  variety  of  environments,  using 
visual,  laser,  sonar,  and  tactile  sensors.  They  have  also  been  incorporated  as  part  of  a 
hand-eye  system,  as  part  of  a  navigation  system  for  autonomous  vehicles,  and  as  part 
of  an  inspection  and  process  control  system  for  industrial  parts.  Recent  efforts  have 
focused  on  establishing  a  formal  theory  on  which  to  judge  the  efficacy  and  robustness  of 
recognition  methods,  on  exploring  alternative  matching  schemes  for  recognizing  objects, 
on  grouping  methods  for  preprocessing  the  input  data  into  salient  sets  of  features,  on  the 
role  of  visual  attention  in  recognition,  and  on  the  use  of  current  recognition  systems  in 
practical  applications. 

Motion  Vision,  Low-Level  Integration,  and  Photogrammetry 

Professor  Horn  and  his  students  worked  on  problems  in  motion  vision.  While  one  can  get 
good  motion  information  from  just  two  image  frames,  distances  to  objects  are  determined 
only  rather  coarsely.  One  difficulty  with  using  many  frames  is  that  one  cannot  generally 
assume  that  the  motion  is  constant  from  frame  to  frame.  It  is  possible,  however,  to 
incorporate  a  dynamic  model  of  the  vehicle  carrying  the  camera  to  constrain  the  likely 
changes  in  motion  over  time.  This  enables  application  of  the  well-known  techniques  of 
Kalman  filtering,  although  the  problems  here  are  highly  non-linear  and  the  equivalent 
“state”  has  an  enormous  number  of  degrees  of  freedom,  typically  one  per  picture  cell, 
which  prevents  application  of  traditional  direct  approaches.  In  a  related  development, 
methods  fiom  computer  graphics  are  used  to  predict  the  shape  and  position  of  an  object 
at  the  next  image  frame  time,  based  on  the  estimated  shape  and  position  and  the  estimated 
motion  at  the  present  time.  Dramatic  improvements  in  the  accuracy  of  the  reconstructed 
object  shape  are  attained  in  this  fashion,  although  after  about  ten  frames  the  errors  in¬ 
troduced  by  the  prediction  phase  begin  to  balance  out  the  improvements  obtained  ftom 
continuing  the  solution  in  time. 

Because  recovery  of  information  about  the  world  from  a  single  cue  such  as  motion 
parallax,  binocular  stereo  disparity,  or  shading  in  images  tends  to  not  be  very  robust,  there 
was  a  great  deal  of  interest  in  integrating  information  from  multiple  cues.  The  intimate 
integration  of  early  vision  modules  will  be  required  for  most  practical  applications  of 
vision  systems.  Professor  Horn’s  approach  to  the  problem  focuses  on  intimate  integration 
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at  the  lowest  level  of  vision  modules.  In  the  simplest  case,  this  means  interlacing  iterations 
of  different  schemes  for  recovering  shape,  or  more  formally,  constructing  a  compound 
functional  that  contains  penalty  terms  for  mismatching  information  available  from  both 
cues  being  considered.  Preliminary  results  in  integrating  motion  vision  and  shape  from 
shading,  and  in  integrating  binocular  stereo  and  shape  from  shading  show  great  promise. 

Visual  Motion  and  Human  Vision 

Professor  Hildreth’s  research  addressed  the  analysis  of  visual  motion  with  special  empha¬ 
sis  on  biologically  plausible  theories.  Her  work  in  the  final  year  of  the  contract  focused 
on  the  recovery  of  the  three-dimensional  motion  and  structure  of  objects,  and  followed 
three  directions.  The  first  is  the  computation  of  qualitative  or  partial  information  regard¬ 
ing  3-D  structure  and  motion  for  tasks  such  as  navigation.  She  developed  a  model  that 
uses  simple  estimates  of  time-to-collision,  based  on  the  changing  image  size  of  moving 
objects,  to  reconstruct  their  3-D  trajectories  through  space,  and  is  conducting  perceptual 
experiments  to  test  whether  such  a  strategy  is  used  by  the  human  visual  system.  Sec¬ 
ond,  she  explored  the  integration  of  3-D  structurc-from-motion  recovery  with  the  overall 
process  of  surface  reconstruction.  This  latter  project  has  led  Professor  Hildreth  to  study 
the  interaction  between  motion  analysis  and  binocular  stereopsis.  She  pursued  a  model 
that  computes  the  3-D  positions  and  velocities  of  features  over  an  extended  time,  through 
incremental  improvement,  by  combining  constraints  both  from  the  projected  2-D  motions 
of  the  points  in  the  image  and  the  temporal  changes  in  their  stereo  disparity. 

The  Vision  Machine 

The  main  project  of  Professor  Poggio’s  group  was  the  Vision  Machine— a  computer 
system  that  attempts  to  integrate  several  visual  cues  to  achieve  high  performance  in 
unstructured  environments  for  the  tasks  of  visual  recognition  and  navigation.  The  Vision 
Machine  became  a  test-bed  for  measuring  progress  in  the  theory  of  early  vision  algorithms, 
their  parallel  implementation  and  their  integration,  up  to  recognition  of  3'D  objects. 
They  developed  and  implemented  several  parallel  early  vision  algorithms  computing  edge 
detection,  stereo,  motion,  texture,  and  color  in  close  to  real  time.  The  integration  stage 
attempts  to  derive  a  map  of  the  surface  discontinuities  in  the  scene,  with  a  partial  labeling 
of  the  intensity  edges  in  terms  of  their  physical  origin.  They  interfaced  the  output  of 
their  integration  stage  with  a  parallel  recognition  algorithm. 

During  the  final  year  of  the  contract,  one  of  the  main  achievements  was  the  design 
and  fabrication  of  an  analog  VLSI  chip  embedding  one  of  the  earliest  Vision  Machine 
algorithms — edge  detection — integrated  with  a  CCD  imager  on  the  same  chip. 
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Hardware  and  Software 


Message-Passing  Semantics 

The  Message-Passing  Semantics  group,  under  the  guidance  of  Professor  Hewitt,  devel¬ 
oped  the  foundations  for  Open  Systems  that  perform  robusdy  in  changing  environments. 
An  Open  System  is  one  that  is  always  subject  to  unanticipated  communications  from 
outside  and  whose  operations  are  subject  to  indeterminate  results.  Robusmess  means  the 
ability  to  keep  commitments  in  the  face  of  conflict  and  indeterminacy,  which  are  ubiq¬ 
uitous  in  Open  Systems.  Robust  computer  systems  are  needed  to  meet  the  challenge  of 
Open  Systems  to  gain  from  the  advantages  of  openness  while  meeting  the  requirements 
that  are  imposed  by  openness.  Open  Systems  undergo  continual  change:  some  change 
coming  from  within,  through  communication  among  internal  parties,  some  from  with¬ 
out  through  interaction  with  the  environment  The  primitives  of  ultraconcurrent  systems 
are  called  ACTORs.  These  can  be  organized  into  systems  of  ORGs  (Organizations  of 
Restricted  Generality).  The  Actor  model  provides  a  scientific  and  technological  basis 
for  Open  Systems  because  it  suppons  dynamic  reconfigurability,  compositionality,  and 
extensibility.  The  ORG  model  provides  a  scientific  and  technological  basis  for  organi¬ 
zational  systems  because  it  supports  teamwork,  management,  liaison,  and  organizational 
representation.  The  group’s  research  focuses  on  theoretical,  architectural,  and  linguistic 
aspects  of  organizational  systems  composed  of  humans  and  telecomputer  systems. 

Symbolic  Parallel  Architectures 

The  Symbolic  Parallel  Architecture  group,  under  the  direction  of  Professor  Knight,  has 
developed  a  design  for  a  uniform,  large  scale,  parallel  symbolic  supercomputer  called 
Transit.  Unlike  most  parallel  machines,  this  architecture  has  been  explicitly  designed  to 
support  a  wide  range  of  parallel  programming  models  with  excellent  performance.  The 
key  realization  is  the  critical  importance  of  low  latency  in  the  processor-to-processor 
communications  path.  This  low  latency  communications  is  used  as  a  substrate  for  coher¬ 
ent  caches  and  processor-to-processor  message  passing.  The  implementation  of  Transit, 
now  underway,  will  be  done  in  three  phases:  construction  of  the  routing  network,  coher¬ 
ent  cache  implementation,  and  finally  processor  design.  The  routing  network  is  currently 
under  detailed  design  and  simulation.  Its  construction  involves  novel  three-dimensional 
packaging  and  cooling  technology,  novel  VLSI  techniques  for  chip-to-chip  communica¬ 
tions,  and  a  very  simple,  high  speed  routing  component.  The  initial  prototype  is  exp)ected 
to  yield  a  remote  memory  access  latency  of  about  300ns  and  a  p)er-px>n  peak  bandwidth 
of  800  megabaud  The  aggregate  switch  bandwidth  approaches  a  terabaud 
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Analogical  and  Formal  Reasoning 

Learning  from  Analogous  Precedents 

Professor  Winston’s  group  concentrated  on  developing  representations  that  enable  learn¬ 
ing  and  reasoning  by  analogy.  During  the  final  year  of  the  contract,  considerable  progress 
was  made  on  the  panicular  problem  of  representing  decision  rationals.  The  key  problem 
was  to  find  a  suitable,  framelike  vocabulary  for  describing  decisions.  By  capturing  a  deci¬ 
sion  rational  in  a  form  that  facilitates  symbolic  reasoning,  subsequent  decisions  need  not 
go  over  the  same  ground.  Additionally,  decision  can  be  seen  from  a  variety  of  perspec¬ 
tives,  providing  a  sort  of  symbolic  what-if  capability,  leading  to  a  better  understanding 
of  how  biases  influence  conclusions. 

Other  work  in  Professor  Winston’s  group  has  focused  on  the  problem  of  representing 
change  qualitatively,  such  that  a  remembered  sequence  of  changes  can  be  used  as  a 
precedent  for  understanding  how  some  subsequent  situation  is  evolving.  The  experimental 
domain  was  that  of  encyclopedia-style  texts  describing,  for  example,  how  rockets  work. 

Automated  Formal  Reasoning 

Professor  McAllester  concentrated  on  building  and  testing  automated  formal  reasoning 
systems.  These  formal  reasoning  systems  incorporate  a  variety  of  new  algorithmic  tech¬ 
niques  that  allow  effecdve  automated  reasoning  about  topics  that  are  beyond  the  scope  of 
any  previous  reasoning  system.  For  example,  the  new  reasoning  systems  have  been  able 
to  verify  proofs,  starting  with  only  the  axioms  of  Zermelo-Fraenkel  set  theory,  of  the 
Stone  representation  theorem  in  lattice  theory.  This  theorem  involves  an  ultrafilter  con¬ 
struction  and  is  similar  in  complexity  to  the  Tychonoff  theorem  that  a  product  of  compact 
topological  spaces  is  compact.  The  novel  algorithmic  techniques  include  the  integration  of 
congruence  closure  into  general  theorem  proving,  monotone  closure  for  reasoning  about 
semantic  types,  focused  forward  chaining,  and  the  incorporation  of  universal  generaliza¬ 
tion  into  constraint  propagation.  In  addidon  to  evaluating  automated  reasoning  systems 
in  terms  of  their  ability  to  verify  abstract  matl^madcal  theorems.  Professor  McAllester 
has  studied  the  applicadon  of  automated  reasoning  systems  in  software  verificadon.  In 
pardcular,  during  the  final  year  of  the  contract.  Professor  McAllester  began  to  concentrate 
on  the  special  case  of  verifying  computer  programs  to  be  “uncrashable.” 
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